Many computer systems rely upon expansion busses to add functionality to the overall system. Generally, the added functionality takes the form of small printed circuit boards or other types of ‘cards’ that have on them the necessary components to allow the main processor to communicate with other devices. For example, video cards, audio cards, and network interface cards all provide added functionality to the system. The cards may communicate over an expansion bus, rather than being included on the main system bus.
Expansion busses are generally one of two types, an ISA (Industry Standard Architecture) bus or a PCI (Peripheral Component Interconnect) bus. The ISA standard was used initially, but became a bottleneck as processor speeds increased. Typically, most computer systems now employ PCI busses, or busses similar to PCI-X (PCI eXtended). The device that connects the PCI bus to the main system bus is usually referred to as a PCI bridge.
Expansion cards communicate with the CPU across the expansion bus. When the CPU needs an expansion device, such as a network interface card, to transmit data, it sets up a transmit ring in the memory to direct the device to the data to be transmitted by writing data descriptors to the transmit ring. The CPU then writes to a device control register set up in the system memory for the expansion device that will transmit the data. When the CPU wants to notify the device of its pending task, it will do so through the PCI bridge. The device then fetches one or more of the descriptors in a single PCI transaction (PCI burst) and then generally one packet at a time until the entire data to be transmitted is fetched. The device then transmits the data as requested by the CPU.
PCI bridges may ‘read ahead’ of a transaction, or ‘prefetch’ data from the system memory with the idea being that having the data available to the device at the PCI bridge, rather than in the system memory, would speed the process. Unfortunately, the bridge does not have a good estimate of how much data to prefetch. Bridges may end up prefetching too much data and having to discard the data. Prefetching of the data occupies the bus and the bridge, and wasting any data prefetched reduces the overall system efficiency.
This can lead to a high load on the PCI bus and the device, as well as slowing down the speed of transmission through the interface. It may also place a high load on the system memory, which in turn can slow down the effective speed of the CPU. This problem is compounded when another expansion bus is added to the system. When multiple busses exist in a system, there may be PCI bridges that bridge between the busses. These types of bridges are often referred to a PCI-to-PCI bridges, or P2P bridges. For ease of discussion, the term PCI bridge will be used to refer to both PCI and P2P bridges.